Texture synthesis involves automatically generating large textures from a small example image (known as a texture sample or exemplar). This exemplar-based texture synthesis takes an exemplar and generates additional content based on that exemplar to create much more content than is contained in the exemplar. Exemplar-based texture synthesis alleviates the need to store explicitly an entire expanse of content. Instead, the content is generated “on the fly” from the much smaller exemplar.
Traditionally, exemplar-based texture synthesis includes a correaction process that compares neighborhoods of each synthesized pixel with neighborhoods of the exemplar. The synthesized pixels then are modified to recreate synthesized neighborhoods that are compatible with the exemplar content. For each pixel, the best matching neighborhood is found, and then the current pixel is replaced with the best-matching pixel. The best-matching neighborhood is determined by comparing pixel colors in a small grid of pixels. For example, for each pixel, a 5×5 neighborhood of pixels may be examined, and the error of the neighborhood is the sum of the errors of the red, green, and blue (RGB) color vectors of individual pixels in the 5×5 neighborhood.
Texture synthesis can be isometric or anisometric (meaning “not isometric”). Isometric texture synthesis generates texture having the same pattern at the same scale and at the same orientation. On the other hand, anisometric texture synthesis allows variation in the orientation and local scaling of features in a generated texture.
During the correaction process, a neighborhood is gathered in the currently synthesized texture and compared with neighborhoods from the exemplar. In order to perform anisometric synthesis, current techniques deform the gathered neighborhood. An inverse Jacobian is the transformation applied to a neighbor's coordinates before fetching their color. The inverse Jacobian is a 2×2 matrix that transforms the pixel neighborhood using an affine transformation. In particular, given a Jacobian field, a pixel neighborhood is transformed by the inverse Jacobian. In order to compensate for the inverse Jacobian, the orientation and scale of the pixel neighborhood is changed. This means that rather than retrieving colors at the original points of the neighborhood, colors are retrieved at a new set of transformed neighborhood points, due to the new orientation and scaling.
There are several drawbacks to these current anisometric texture synthesis techniques. The first drawback is lack of memory coherency. In particular, in some situations the transformed neighborhood becomes large and spread out. This means that pixels a great distance away may be accessed instead of immediate pixel neighbors. This leads to more random memory accesses, which can cause inefficiency and lack of coherency in the memory access.
A second drawback is the need to resample non-local pixels that are accessed during the correaction process. Specifically, after transformation the pixels have to be resampled by some process that interpolates their values. This is necessary because the transformed pixels are spread out over large area. However, this resampling process is inefficient and can cause visible blurring in the synthesized texture.
A third drawback is lack of texture coherence. Because the samples are widely spaced apart, the correaction process no longer depends on the values of adjacent pixels. This can drastically decrease texture coherency. This incoherency can make the correaction process behave poorly and ineffectively, and can even cause the process to fail to converge.
Another current anisometric texture synthesis technique is to represent several versions of the exemplar under a variety of rotations and scales, and to select among these exemplar versions based on the desired Jacobian at the current pixel. However, this approach requires significant memory usage to store and analyze the many exemplar versions. Moreover, texture coherency is also difficult to achieve.